Unsupervised Discovery of Persian Morphemes
نویسندگان
چکیده
This paper reports the present results of a research on unsupervised Persian morpheme discovery. In this paper we present a method for discovering the morphemes of Persian language through automatic analysis of corpora. We utilized a Minimum Description Length (MDL) based algorithm with some improvements and applied it to Persian corpus. Our improvements include enhancing the cost function using some heuristics, preventing the split of high frequency chunks, exploiting penalty for first and last letters and distinguishing pre-parts and post-parts. Our improved approach has raised the precision, recall and f-measure of discovery by respectively %32, %17 and %23.
منابع مشابه
Unsupervised Concept Discovery In Hebrew Using Simple Unsupervised Word Prefix Segmentation for Hebrew and Arabic
Fully unsupervised pattern-based methods for discovery of word categories have been proven to be useful in several languages. The majority of these methods rely on the existence of function words as separate text units. However, in morphology-rich languages, in particular Semitic languages such as Hebrew and Arabic, the equivalents of such function words are usually written as morphemes attache...
متن کاملAccuracy Order of Grammatical Morphemes in Persian EFL Learners: Evidence for and against UG
This study addresses the acquisition of the morphological markers in Persian learners of English as a foreign language. To this end, the accuracy order of nine morphemes including plural –s, progressive –ing, copula be, auxiliary be, irregular past tense, regular past tense –ed, third person –s, possessive -ʼs and indefinite articles was studied in 6...
متن کاملUnsupervised Morpheme Discovery with Ungrade
In this paper, we present an unsupervised algorithm for morpheme discovery called UNGRADE (UNsupervised GRAph DEcomposition). UNGRADE works in three steps and can be applied to languages whose words have the structure prefixes-stem-suffixes. In the first step, a stem is obtained for each word using a sliding window, such that the description length of the window is minimised. In the next step p...
متن کاملPersian Computational Morphology: A Unification-Based Approach
This report provides a complete descriptive analysis of Persian inflectional morphology from a computational perspective. The parts of speech and the morphemes that appear on them as well as their corresponding morphotactics are presented in detail. The verbal paradigm is also described in this document. Since the morphological analyzer designed for this project uses a unification-based grammar...
متن کاملUnsupervised Morphemes Segmentation
In this work, we describe the algorithm adopted to split the words into smallest possible meaningful units or morphemes. The algorithm is unsupervised and not dependent on any language. The model is developed using English language. However, the linguistic rules specific to English language are not implemented. The algorithm focuses on the identification of smallest units of words based on thei...
متن کامل